Skip to content

feat: Show all crawl runs + crawl deletion UX improvements #2917

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 21 commits into from
Nov 5, 2025
Merged

feat: Show all crawl runs + crawl deletion UX improvements #2917

merged 21 commits into from
Nov 5, 2025

Conversation

@SuaYoo
Copy link
Member

@SuaYoo SuaYoo commented Oct 22, 2025 •
edited
Loading

Resolves #2906

Changes

Overall, a more consistent experience for navigating all workflow crawls and managing crawl-related items.

  • Adds new "Crawl Runs" page under "Crawling" to display all crawl runs, including canceled and failed
  • Enables sorting crawls for by execution time and page count
  • Updates icon for "Canceled" state to not appear as error
  • Update archived item type labels
  • Refactors archived item and crawl deletion confirmation to use same dialog (partially addresses Switch to in-page dialogue boxes #619)
  • Show collections in deletion confirmation
  • Refactors /archived-items/crawl-list.ts component to -> /crawls/crawl-list/ components

Manual testing

Note: New sort fields require building the backend locally.

  1. Log in as crawler
  2. Go to "Crawling". Verify "Workflows" and "Crawl Runs" tabs are shown
  3. Go to "Crawl Runs"
  4. Interact with search, sort, and filters. Verify list controls work as expected.
  5. Choose canceled crawl run. Verify navigation to crawl details view
  6. Choose action menu -> "Delete Crawl". Verify that crawl is deleted without confirmation
  7. Go back to "Crawl Runs"
  8. Choose successful crawl.
  9. Choose action menu -> "Delete Crawl". Verify that confirmation is shown.
  10. Repeat 9 from "Workflow" > "Crawls" and "Archived Items". Verify confirmation dialog is consistent for all crawl/item deletions.

Screenshots

Page Image/video
Crawl Runs Screenshot 2025-10-21 at 5 36 45 PM
Crawl Runs Screenshot 2025-10-21 at 5 38 51 PM
Workflow / Crawls Screenshot 2025-10-21 at 5 38 23 PM
Archived Items Screenshot 2025-10-21 at 6 12 37 PM
Delete Confirmation (when in collections) Screenshot 2025-10-21 at 6 14 50 PM

Follow-ups

A tag filter can be added once #2923 is merged.

@SuaYoo SuaYoo changed the title wip: Show all crawl runs feat: Show all crawl runs Oct 22, 2025
@SuaYoo SuaYoo changed the title feat: Show all crawl runs feat: Show all crawl runs + crawl deletion UX improvements Oct 22, 2025
@SuaYoo SuaYoo marked this pull request as ready for review October 22, 2025 01:18
@SuaYoo SuaYoo requested review from emma-sg, ikreymer and tw4l October 22, 2025 01:19
@tw4l

This comment was marked as resolved.

Sign in to view
@SuaYoo

This comment was marked as resolved.

Sign in to view
@tw4l

This comment was marked as resolved.

Sign in to view
@SuaYoo
Copy link
Member Author

SuaYoo commented Oct 23, 2025

This is in Brave, and there is a horizontal scrollbar to scroll the table, so maybe it's okay after all. I think I was just expecting the row to collapse like the workflows, but now I see many other elements in Browsertrix handle smaller screens like this table.

It's a good call out, and I'll do a quick pass to address the inconsistent text wrapping at different screen sizes. Will link your comments to the general responsiveness improvements issue: #1348

@SuaYoo SuaYoo mentioned this pull request Oct 23, 2025
Open
@SuaYoo SuaYoo force-pushed the frontend-crawling-subsections branch from 084133c to 07da671 Compare October 27, 2025 16:53
@SuaYoo SuaYoo force-pushed the frontend-crawling-subsections branch from 999dabd to f516d6a Compare November 3, 2025 17:00
tw4l
tw4l approved these changes Nov 3, 2025
View reviewed changes
Copy link
Member

@tw4l tw4l left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested again and working great

emma-sg
emma-sg approved these changes Nov 4, 2025
View reviewed changes
Copy link
Member

@emma-sg emma-sg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One fairly small suggestion, but otherwise looks pretty good!

A few more thoughts:

  • The tag filter feels like it's missing from the crawl runs list controls — I'm not sure how useful it really is, but I think if it's not hard to add it'd be a nice addition
  • I do still think we should convert the tabs on the archived items page into a filter now that we're doing this, but that could always come later
  • The archived item list still needs an overhaul so it can work at smaller viewport sizes — I know Tessa also brought up scroll overflow vs wrapping as a solution as well, probably worth more discussion (but not super high priority either)

frontend/src/pages/org/crawls.ts Outdated
private renderControls() {
return html`
<div class="flex flex-wrap items-center gap-2 md:gap-4">
<div class="grow basis-1/2">${this.renderSearch()}</div>
Copy link
Member

@emma-sg emma-sg Nov 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a small issue where at viewport widths wide enough to reach the max content width, there's a re-layout that happens when selecting any filter that I think could be avoided by increasing the basis here a little more:

Screen.Recording.2025-11-04.at.6.30.48.PM.mov

Copy link
Member Author

@SuaYoo SuaYoo Nov 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting, I didn't see this in Firefox, just reproduced in Chrome.

@SuaYoo
Copy link
Member Author

SuaYoo commented Nov 5, 2025 •
edited
Loading

The tag filter feels like it's missing from the crawl runs list controls — I'm not sure how useful it really is, but I think if it's not hard to add it'd be a nice addition

Agree, we could add after #2923 and before release.

I do still think we should convert the tabs on the archived items page into a filter now that we're doing this, but that could always come later

Created issue here: #2956

The archived item list still needs an overhaul so it can work at smaller viewport sizes — I know Tessa also brought up scroll overflow vs wrapping as a solution as well, probably worth more discussion (but not super high priority either)

Tracking here: #1348

@SuaYoo SuaYoo merged commit 357cbf4 into main Nov 5, 2025
29 checks passed
@SuaYoo SuaYoo deleted the frontend-crawling-subsections branch November 5, 2025 23:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@emma-sg emma-sg emma-sg approved these changes

@tw4l tw4l tw4l approved these changes

@ikreymer ikreymer Awaiting requested review from ikreymer

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Feature]: Show all crawl runs

4 participants

@SuaYoo @tw4l @emma-sg